Detect and quarantine wedged vGPU VFs - #435

Detection moves from host-side matching of the raw NVRM kernel line to an
explicit guest-to-host report: the guest agent watches /dev/kmsg for the
driver's RmInitAdapter failure and emits a HYPEMAN-GPU-INIT-FAILED marker
over the established sentinel-marker channel, which the controller convicts
on. The raw kernel line is no longer matched, so detection now requires the
guest agent; images without it do not report.

Quarantine hardening in the same pass:

- The conviction brake pauses instead of drops: a suppressed conviction
  leaves the tail open and the agent's re-emission retries it once the
  window clears.
- A match on an already-quarantined VF is not a new conviction: no brake
  accounting, no metric, no wedge-count inflation on controller restarts.
- Tails reset when the instance acquires a new assignment, so a finished
  tail from a previous boot or VF cannot suppress scanning the next one.
- A vf-health state file that fails to load refuses mutations (and retries
  the load) instead of letting the next conviction clobber every previously
  persisted quarantine.
- Oversized unterminated log lines are skipped instead of re-buffered on
  every scan.
- The controller idles on hosts without the vendor VFIO framework.

GPU.md: DCGM quiesce is now an ordered step of the recovery sequence, and
clearing vf-health.json documents the immediate-restart requirement.

Review fixes on the quarantine layer:

- start now archives the previous boot's serial log before persisting the
  new vGPU assignment. The sentinel keys its tail on the assignment epoch,
  so the old ordering let a scan replay the previous boot's wedge report
  against the freshly assigned VF. A failed archive is fatal for GPU
  instances instead of a warning.
- placement and profile availability refuse to run when the VF health
  state file exists but cannot be loaded, instead of treating the empty
  in-memory set as healthy and returning every quarantined VF to
  rotation. The load is retried on each attempt, so a repaired file
  self-heals.
- a conviction whose persist fails is rolled back from memory: keeping it
  made the next report look like a repeat conviction, ending retries with
  nothing on disk.
- the marker pattern requires the full shape through the quoted NVRM
  payload; a truncated or payload-less marker echoed by an exec command
  no longer matches.
- unreadable instance metadata in the sentinel target listing logs a
  warning instead of silently shrinking detection coverage.
- GPU.md recovery runbook unwound a circularity: placement excludes
  quarantined VFs and there is no VF-pin API, so the entry is cleared
  before the verification boot; the sentinel re-quarantines automatically
  if the cycle did not cure the VF.

The /dev/kmsg record priority encodes facility*8+level. Kernel printk is
always facility 0 and the kernel assigns userspace writers LOG_USER or
higher (a facility-0 prefix is coerced to LOG_USER, verified on a live
6.12 kernel), so requiring facility 0 makes in-guest forgery of the
report impossible, matching the intended kernel-records-only semantics.

Also retry a failed /dev/kmsg open instead of permanently disabling the
watcher for the guest's lifetime.

The scan buffered each complete line whole, so guest console output
could make the controller allocate line-sized buffers every pass; the
64KB cap only kept an unterminated tail from being re-read, and once
such a line was skipped its late-arriving tail was parsed as a fresh
line. Read through a fixed-size buffer instead: a line that overflows
it cannot be a marker, so it is discarded — across scans if its newline
has not arrived — without ever being held in memory, and its tail can
no longer replay a marker. Rotation resets the skip state with the
offset.

When the persisted state file fails to load, quarantine mutations are
refused and vGPU placement fails closed, but the quarantined-VFs gauge
reads zero from the empty in-memory set — exactly when quarantines
exist and are unreadable. Export the load-failure state as its own
gauge so the condition is alertable.

Auto-conviction no longer pauses on a burst. Systemic non-wedge init
failures (e.g. a driver-mismatch image rollout) are expected to be
caught on a test host before reaching production, and the convictions
counter remains the alerting signal if one gets through; the brake was
extra state and logic guarding against a case the rollout process
already covers. Quarantine still only removes capacity and never
touches instances, and the store's fail-closed load handling is
unchanged.

Each /dev/kmsg read returns exactly one record and fails with EINVAL —
without consuming the record — when the buffer is smaller. Records run
up to CONSOLE_EXT_LOG_MAX (8 KiB), so bufio's default 4 KiB buffer
wedged the watcher on the first oversized record: every reopen replayed
the ring into the same record, silently losing all detection behind it.
Size the buffer to the kernel's record bound and log non-EPIPE scan
errors so a wedge is visible instead of silent.

A quarantine is only real once it is on disk, but the persist renamed
without syncing the file or directory, so a host crash right after a
conviction could silently drop it. Sync the temp file before the rename
and the directory after, and deduplicate the sorted record listing.

The matched line is guest-controlled console bytes up to the 64 KiB
line cap; logging and persisting it verbatim put up to that much guest
output in error logs and vf-health.json. Keep just the marker match.

ClearVFQuarantine had no callers — the runbook documents editing
vf-health.json directly — and WedgeCount was always 1. QuarantineVF's
record return and IsVFQuarantined only served tests, and the sentinel's
isQuarantined pre-check duplicated what QuarantineVF already reports
via existed. The hostFramework indirection wrapped a single
DiscoverVGPU call nothing injected.